1. Field of the Invention
The invention relates to a method and device for sharpness enhancement.
In scenarios where TV and PC systems merge their functionality, especially in the home environment, high-quality displaying of video images in PC architectures becomes a challenging issue for industries oriented to the consumer market. Since the spatial resolution of synthetic images of the PC outperforms that of natural video scenes, the desire for improved quality in TV images will increase. The subjective attribute sharpness, which is determinant by the human visual system, is one of the most important factors in the perception of image quality. Despite many algorithms for 2-D sharpness enhancement have already been proposed, their effectiveness drops in this context. In fact, in the image processing literature, it is often implicitly assumed that operators that have been designed for the enhancement of 2-D data can be straightforwardly applied to video sequences too. As far as contrast sharpening is concerned, this is far from true: conventional 2-D techniques introduce many small artifacts and non-homogeneities which are perfectly acceptable in a still picture but become very visible and annoying in an image sequence. Moreover, the visibility of a defect strongly depends on the sequence contents, namely, on the amount of details and of motion.
Moreover, even more attention has to be paid, in the PC environment, to the case of decompressed images like MPEG, AVI, etc. In such cases, indeed, network and bus throughput constraints and limited storage capacity impose the use of compression techniques, often operating at relatively low bit rates, which cause visible blocking effects in the decompressed images. The artificial high frequencies arising along the borders of the blocks themselves are usually amplified by sharpness enhancement algorithms, so that the blocking artifact would be emphasized.
The task of image enhancement techniques is often to emphasize the details of a scene so as to make them more visible to a human viewer or to aid some machine performance (e.g., object identification); at the same time, they should reduce noise or, at least, avoid its amplification. Algorithms for contrast enhancement are often employed in an interactive fashion with the choice of the algorithm and the setting of its parameters being dependent on the specific application on hand.
2. Description of the Related Art
A large number of approaches have been devised to improve the perceived quality of an image (See Ref. [1]). The histogram equalization, a commonly used method, is based on the mapping of the input gray levels to achieve a nearly uniform output gray level distribution (See Refs. [1], [2]). However, histogram equalization applied to the entire image has the disadvantage of the attenuation of low contrast in the sparsely populated histogram regions. This problem can be alleviated by employing local histogram equalization, which is of high computational complexity.
Another method, called statistical differencing, generates the enhanced image by dividing each pixel value by its standard deviation estimated inside a specified window centered at the pixel (Ref. [2]). Thus, the amplitude of a pixel in the image is increased when it differs significantly from its neighbors, while it is decreased otherwise. A generalization of the statistical difference methods includes the contributions of the pre-selected first-order and second-order moments.
An alternate approach to contrast enhancement is based on modifying the magnitude of the Fourier transform of an image while keeping the phase invariant. The transform magnitude is normalized to range between 0 and 1 and raised to a power which is a number between zero and one (Ref. [3]). An inverse transform of the modified spectrum yields the enhanced image. This conceptually simple approach, in some cases, results in unpleasant enhanced images with two types of artifacts: enhanced noise and replication of sharp edges. Moreover, this method is of high computational complexity.
A simple linear operator that can be used to enhance blurred images is Unsharp Masking (UM) (Ref. [1]). The unsharp masking approach exploits a property of the human visual system called the Mach band effect. This property describes the visual phenomenon that the difference in the perceived brightness of neighboring region depends on the sharpness of the transition (Ref. [1]); as a result, the image sharpness can be improved by introducing more pronounced changes between the image regions. The fundamental idea of UM is to subtract from the input signal a low-pass filtered version of the signal itself. The same effect can, however, be obtained by adding to the input signal a processed version of the signal in which high-frequency components are enhanced; we shall refer to the latter formulation, schematically shown in FIG. 1. FIG. 1 shows that a linearly high-pass filtered (HPF) version of the input signal x(n) is multiplied by a factor λ and thereafter added to the input signal x(n) to form the output signal y(n). The output of the high-pass filter HPF introduce an emphasis which makes the signal variations more sharp. Its effect is exemplified in FIG. 2. In FIG. 2, curve “a” shows the original signal, curve “b” shows a first order derivative of the input signal, curve “c” shows the second order derivative of the input signal, and curve “d” shows the original signal minus k times the second order derivative.
In one dimension, the UM operation, shown by the block diagram in FIG. 1, can be represented mathematically as:y(n)=x(n)+λz(n); where y(n) and x(n) denote the enhanced signal and the original signal, respectively, z(n) means the sharpening component, and λ is a positive constant. A commonly used sharpening component is the one obtained by a linear high-pass filter which can be, for example, the Laplacian operation given byz(n)=2x(n)−x(n−1)−x(n+1) 
Even though this method is simple to implement, it suffers from two drawbacks that can significantly reduce its benefits. First, the operator introduces an excessive overshoot on sharp details. Second, it also enhances the noise and/or digitization effect. The former problem comes from the fact that the UM method assigns an emphasis to the high frequency components of the input, amplifying a part of the spectrum in which the SNR (signal-to-noise ratio) is usually low. On the other hand, wide and abrupt luminance transitions in the input image can produce overshoot effects; these are put into further evidence by the human visual system through the Mach band effect.
Several variants of the linear UM technique have been proposed in literature, trying to reduce the noise amplification. A quite trivial approach consists in substituting a bandpass filter for the high-pass one in FIG. 1. This reduces noise effect, but also precludes effective detail enhancement in most images.
In more sophisticated approaches, Lee and Park (Ref. [4]) suggest using a modified Laplacian in the UM scheme. They propose the order statistic (OS) Laplacian; its output is proportional to the difference between the local average and the median of the pixel in a window. They demonstrate that the resulting filter introduces a much smaller noise amplification than the conventional UM filter, with comparable edge-enhancement characteristics. A different approach is taken in Ref. [5]; in fact they replace the Laplacian filter with a very simple operator based on a generalization of the so-called Teager's algorithm. An example of such an operator is the simple quadratic filter given byz(n)=x2(n)−x(n−1)x(n+1). 
It can be shown that this operator approximates the behavior of a local mean-weighted high-pass filter, having reduced high-frequency gain in dark images areas. According to Weber's law (Ref. [1]), the sensitivity of the human visual system is higher in dark areas; hence, the proposed filter yields a smaller output in darker areas and, therefore, reduce the perceivable noise. Even though the above quadratic operators do take into account Weber's law, their direct use in unsharp masking may still introduce some visible noise depending on the enhancement factor (1 in FIG. 1) chosen.
In order to improve the performance of the UM, in Ref. [6], the output of the high-pass filter is multiplied by a control signal obtained from a quadratic edge sensors:z(n)=[x(n−1)−x(n+1)]2[2x(n)−x(n−1)−x(n+1)]  (1.1) Its purpose is to amplify only local luminance changes due to true image details. The first factor on the right-hand side of Eq. 1.1 is the edge sensor. It is clear that the output of this factor will be large only if the difference between x(n−1) and x(n+1) is large enough, while the squaring operation prevents interpreting small luminance variations due to noise as true image details. The output of the edge sensor acts as a weight for the signal coming from the second factor in Eq. 1.1, which is a simple linear high-pass filter.
Another nonlinear filter, the Rational UM technique, has been devised (Ref. [7]): the output of the high-pass filter is multiplied by a rational function of the local input data:       z    ⁡          (      n      )        =                                          [                                          x                ⁡                                  (                                      n                    -                    1                                    )                                            -                              x                ⁡                                  (                                      n                    +                    1                                    )                                                      ]                    2                                                    k              ⁡                              [                                                      x                    ⁡                                          (                                              n                        -                        1                                            )                                                        -                                      x                    ⁡                                          (                                              n                        +                        1                                            )                                                                      ]                                      4                    +          h                    ⁡              [                              2            ⁢                          x              ⁡                              (                n                )                                              -                      x            ⁡                          (                              n                -                1                            )                                -                      x            ⁡                          (                              n                +                1                            )                                      ]              .  
In this way, details having low and medium sharpness are enhanced; on the other hand, noise amplification is very limited and steep edges, which do not need further emphasis, remain almost unaffected. Under a computational viewpoint, this operator maintains almost the same simplicity as the original linear UM method.
A similar approach is also proposed in Ref. [8]; the method is similar to the conventional unsharp masking structure, however, the enhancement is allowed only in the direction of maximal change and the enhancement parameter is computed as a rational function like to the one described in Ref. [7]. The operator enhances the true details, limits the overshoot near sharp edges and attenuates noise in at areas. Moreover, it is applied to color image enhancement by using an extension of the gradient to multi-valued signal.
Finally, in Ref. [9], the unsharp masking technique is extended with an advanced adaptive control that uses the local image content. More precisely, they have concentrated on the following properties for adaptive control of the sharpness enhancement:                local intensity level and related noise visibility;        noise level contained by the signal;        local sharpness of the input signal;        aliasing prevention, where alias results from nonlinear processing such as clipping.        
These four properties of the video signal are analyzed locally by separate units and the amount of the sharpness enhancement depending on this analysis.
All the proposed contrast enhancement methods have been designed for the enhancement of 2-D data and cannot be applied to video sequences. In fact, they introduce many small artifacts and non-homogeneities which are perfectly acceptable in a still picture but become very visible and annoying in an image sequence. Moreover, conventional 2-D techniques cannot be applied to the enhancement of block-coded image sequences, because they emphasize such artifacts that become very visible. The local spectra and bandwidth of both noise and the signal vary spatially, and the characteristics of the filters need to be locally adapted.